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The South Australian Certificate of Education (SACE) is a credential and formal 
qualification within the Australian Qualifications Framework. A recent review of the 
SACE outlined a number of recommendations for significant changes to this 
certificate. These recommendations were the result of a process that began with the 
review panel “scrutinizing carefully [existing SACE structures for] continuing validity 
and effectiveness This paper critiques the “careful examination ” of statistical trends 
and patterns used to build the case for reform. Central to these trends and patterns are 
measures of retention, socio-economic status and student achievement, all of which 
are problematic. This paper also challenges the appropriateness of the statistical 
techniques used in the review. The paper concludes by arguing that making significant 
policy changes based upon such limited and flawed analyses is problematic. 

Educational research, curriculum, education policy, post-compulsory education 


INTRODUCTION 

The South Australian Certificate of Education (SACE), introduced in 1992-93, is recognised as a 
credential and formal qualification within the Australian Qualifications Framework (Keightley 
and Keighley-James, 2001). SACE consists of two stages: Year 11 students typically complete 
Stage 1 and Year 12 students usually complete Stage 2. Offering in excess of 70 subjects over the 
two stages to largely post-compulsory education students, the SACE was recently reviewed by a 
three-person Review Panel (Crafter, Crook, and Reid, 2006). The substantive term of reference 
required the Review Panel to “achieve a curriculum and assessment framework that will meet the 
diverse needs of all students and result in high and more socially equitable levels of retention, 
completion and pathways beyond school” (Crafter et ah, 2006, p. 9). The product of the review, a 
proposed new SACE, represents a substantial shift in “what counts as knowledge, the ways in 
which it is organised, who is empowered to teach it, [and] what counts as an appropriate display 
of having learned it” (Apple, 1993, p. 222). 

Crafter et al. (2006) make a number of recommendations for the creation of a new SACE based 
upon principles, design concepts and features that are, they claim, flexible and responsive, 
credible, inclusive, connected, worthwhile, futures-orientated, and supportive of quality learning 
and teaching. For example, the Review Panel recommends the replacement of statistical 
moderation - the method used to adjust students’ school-based assessment marks based upon 
external examination scores - with a method where “significant differences” between internal and 
external assessments will “signal the need for further discussion with the school and the teacher” 
and “this may result in adjustments to assessments” (Crafter et al., 2006, p. 132). This and other 
recommendations emerged from a process that began with the review panel “scrutinising carefully 
[existing SACE structures for] continuing validity and effectiveness” (Crafter et al., 2006, p. 9). 

The Final Report is divided into three major sections: the case for reform, a new SACE within a 
learning space, and detailed reform proposals. The focus of this paper is the case for reform. In the 
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Final Report, three chapters are used to present the case: current challenges, the case for reform - 
statistical trends and patterns, and the case for reform - community views. This paper examines 
the statistical trends and patterns that were used to build the case for reform. These trends and 
patterns are fundamental to the Review Panel’s case for reform and serve as a primary platfonn 
upon which it bases its recommendations. 

The Review Panel used a “relatively small number of ‘key indicators’ [to] highlight some of the 
more significant trends and patterns in the participation and achievement of young people” 
(Crafter et ah, 2006, p. 3 1). The trends and patterns identified in the review rely largely upon three 
measures: retention rates, socio-economic disadvantage, and achievement scores. The use of each 
measure is explored in detail in this paper. 

FULL-TIME APPARENT RETENTION AND THE SACE REVIEW 

Three overlapping conceptual definitions of early school leavers can be found in the literature 
(Dekkers and Claassen, 2001). First, early school leavers may be defined as those students who 
leave before the end of compulsory-aged schooling. Thus, for example, a South Australian student 
who leaves the education system before he or she is 16-years old would be classified as an early 
school leaver. South Australian students typically reach this age late in Year 10 or early in Year 

11, with smaller numbers reaching that age in Years 9 and 12. Second, early school leavers may 
be defined as those students who leave without obtaining a certificate or diploma. Within the 
South Australian system, they would leave without having satisfactorily completed the 
requirements of the SACE certificate which are typically satisfied at the end of Year 12. The third 
conceptual approach views an early school leaver as a student who leaves school to take up 
employment. Under this conceptualisation, early leaving is problematic only if the leaver does not 
have the minimal levels of education and training necessary to enter the labour market (Hannan, 
Hovels, Berg, and White, 1995). 

Crafter et al. (2006) used a certificate-based definition together with staying at school to the end 
of Year 12. They draw the South Australian Minister of Education’s attention to the apparently 
low SACE completion rate, thereby casting the discussion in terms of students leaving the 
secondary education system without the certificate. They also speak of retention in tenns of 
students, defined as Year 8 or Year 10 cohorts, who stay at secondary school until the end of Year 

12, typically one to two years beyond the end of compulsory schooling in South Australia. In 
adopting this approach, Crafter et al. (2006) are following the convention used by the Australian 
Bureau of Statistics (ABS). The ABS uses four age-based definitions of apparent retention, 
reflecting the imprecise nature of their indices. Adapted to South Australia, the four ABS 
definitions of apparent retention are: 

• Apparent full-time retention is defined as the number of full-time students in Year 12, 
expressed as a percentage of the number who started secondary education in Year 8. 

• Apparent full-time retention can be expressed as the number of full-time students in Year 
12, expressed as a percentage of the number who started schooling in Year 10. 

• Apparent all retention is defined as the number of full-time-equivalent students in Year 12 
expressed as a percentage of the number of Year 8 students. 

• Apparent all retention is defined as the number of full-time-equivalent students in Year 12 
expressed as a percentage of the number of Year 10 students. 

The ABS full-time retention rates, from Year 10 to Year 12, are shown in Table 1. Crafter et al. 
(2006) correctly state that the South Australia full-time retention rate is reasonably static and is 
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the second-lowest rate of the eight regions monitored, although arguably the ACT and NSW 
figures are confounded by a number of factors including the movement of students from NSW to 
ACT. Crafter et al. (2006) use these low full-time retention rates for South Australia as a major 
plank upon which to build their argument for an overhaul of SACE. 


Table 1. Apparent Full-Time Retention Rate, from Year 10 to Year 12 (Source: ABS 
Schools Australia, 2005, Cat. No. 4221.0) 


Year 

Aust. 

ACT 

NT 

Tas 

WA(b) 

SA 

Qld 

Vic. 

NSW 

Government 

1998 

69.4 

110.0 

62.4 

62.3 

67.6 

64.2 

73.0 

73.7 

64.8 

1999 

69.6 

107.1 

70.0 

67.5 

67.8 

64.8 

73.2 

73.5 

64.7 

2000 

69.6 

105.0 

69.9 

71.6 

67.3 

61.9 

73.9 

74.4 

64.2 

2001 

70.6 

112.1 

70.6 

70.5 

67.0 

61.7 

74.8 

76.8 

65.1 

2002 

72.2 

101.0 

73.1 

75.3 

69.7 

61.9 

76.5 

77.5 

67.4 

2003 

71.9 

101.0 

78.7 

76.4 

64.8 

61.8 

76.4 

77.3 

68.1 

2004 

72.2 

100.8 

90.8 

76.5 

66.7 

62.9 

75.0 

77.2 

68.6 

2005 

71.3 

99.5 

76.2 

67.2 

66.3 

64.4 

72.7 

77.0 

68.5 

Non-government 

1998 

83.4 

68.6 

54.8 

69.4 

80.3 

85.2 

86.8 

88.3 

80.5 

1999 

83.6 

72.2 

52.6 

72.7 

79.1 

84.4 

88.0 

87.3 

81.2 

2000 

83.6 

65.2 

43.9 

71.6 

80.2 

84.1 

87.8 

88.3 

81.2 

2001 

84.5 

68.7 

52.1 

70.5 

81.9 

85.0 

89.1 

89.6 

80.9 

2002 

85.9 

75.2 

49.8 

74.1 

82.1 

86.9 

89.8 

91.5 

82.1 

2003 

85.9 

76.3 

46.5 

76.2 

81.9 

87.9 

90.6 

91.8 

81.0 

2004 

86.1 

72.6 

43.1 

75.9 

82.4 

87.7 

91.5 

91.9 

81.3 

2005 

85.4 

74.5 

54.2 

69.2 

82.3 

86.2 

91.2 

90.2 

81.3 

Total 

1998 

74.1 

79.4 

60.4 

64.2 

71.8 

71.2 

77.7 

79.1 

69.8 

1999 

74.4 

92.5 

64.7 

68.9 

71.5 

71.4 

78.3 

78.7 

70 

2000 

74.4 

88.7 

62.2 

71.6 

71.6 

69.5 

78.7 

79.7 

69.8 

2001 

75.4 

93.3 

64.9 

70.5 

71.9 

69.6 

79.7 

81.6 

70.3 

2002 

77 

89.8 

66.2 

75 

73.9 

70.6 

81.1 

82.9 

72.4 

2003 

76.9 

90.3 

68.7 

76.4 

70.6 

70.8 

81.5 

82.9 

72.7 

2004 

77.2 

88.4 

75.2 

76.3 

72.4 

71.6 

80.8 

83 

73.2 

2005 

76.5 

88.1 

69.5 

67.8 

72.2 

72.1 

79.3 

82.2 

73.2 


However, the focus on full-time students is misleading. South Australia has the highest number of 
part-time students of any state or territory (ABS, 2003). In 2002, 20.0 per cent of all South 
Australian Year 12 students were studying part-time (ABS, 2003). This is mainly due to the 
number of students who decide to work part-time and study part-time. It is also a reflection of 
study patterns emerging in the SACE itself. In some schools, it is relatively common for students 
to start their SACE studies in Year 10 and take a reduced load in Year 12, thus spreading out a 
two-year program over three years. Such students are technically part-time students in their final 
year of secondary education, even though they are in effect fully engaged with SACE. For 
example, S SABS A reports that 3,597 students from independent schools fulfilled the 
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requirements for entry into a South Australian university in 2005 (SSABSA 2005) and yet the 
ABS (Cat. No. 4221.0, Table 9, 2005) reports that there were only 2,947 full-time Year 12 
students in those schools in this year. The ABS, aware of these and other patterns, issues the 
caution: “because of this pattern of education in SA, it is important to look at the all students [i.e., 
full-time-equivalent] retention rate as well as the full-time rate” (ABS, 2003). As shown in Table 
2, when the full-time-equivalent retention rate is used, South Australia’s apparent retention rate is 
above the national average, and slightly smaller than Tasmania and Australian Capital Territory. 

Table 2. Apparent Full-Time-Equivalent Retention Rates, from Years 10 to Year 12 


(Source: ABS Schools Australia, 2005, Cat. No. 4221.0) 


Year 

Aust. 

ACT 

NT 

Tas 

WA 

SA 

Qld 

Vic. 

NSW 

Government 

1998 

74.4 

110.4 

82.2 

87.2 

73.3 

85.3 

73.2 

76.4 

68.4 

1999 

75.0 

107.3 

93.3 

104.2 

73.2 

84.8 

73.4 

76.5 

68.6 

2000 

75.3 

105.2 

79.9 

102.1 

71.8 

84.1 

76.5 

77.3 

68.1 

2001 

76.4 

112.2 

87.4 

107.0 

73.4 

83.1 

77.0 

80.4 

67.9 

2002 

78.5 

101.3 

85.9 

119.7 

74.2 

86.4 

79.8 

81.2 

70.1 

2003 

77.5 

102.7 

88.3 

122.6 

65.3 

83.7 

79.0 

81.0 

71.4 

2004 

77.6 

101.8 

99.7 

125.9 

68.3 

86.1 

77.0 

80.9 

71.6 

2005 

76.1 

100.8 

84.4 

102.2 

67.7 

85.0 

74.7 

80.5 

71.1 

Non-Government 

1998 

84.0 

68.6 

54.8 

69.6 

80.4 

89.1 

86.9 

88.6 

81.2 

1999 

84.1 

72.2 

55.2 

72.9 

79.2 

87.6 

88.1 

87.7 

81.7 

2000 

84.1 

65.2 

46.2 

72.2 

80.3 

87.3 

87.9 

88.6 

81.6 

2001 

85.0 

68.7 

53.8 

71.2 

81.9 

88.5 

89.2 

89.8 

81.4 

2002 

86.4 

75.3 

52.1 

74.4 

82.1 

90.8 

89.9 

91.8 

82.6 

2003 

86.5 

76.4 

48.7 

75.6 

81.9 

92.8 

90.8 

92.0 

81.6 

2004 

86.6 

72.8 

45.0 

75.3 

82.7 

92.8 

91.7 

92.0 

81.8 

2005 

86.0 

75.6 

54.8 

69.3 

82.4 

91.0 

91.4 

90.4 

81.7 

Total 

1998 

77.6 

92.7 

74.9 

82.4 

75.6 

86.5 

77.7 

80.9 

72.5 

1999 

78.1 

92.6 

82.0 

95.7 

75.2 

85.7 

78.2 

80.7 

72.8 

2000 

78.3 

88.9 

70.2 

94.0 

74.6 

85.2 

80.4 

81.6 

72.6 

2001 

79.4 

93.3 

77.3 

97.0 

76.2 

84.9 

81.2 

84.0 

72.4 

2002 

81.3 

90.0 

76.3 

107.2 

76.9 

87.9 

83.3 

85.3 

74.4 

2003 

80.7 

91.3 

76.4 

109.4 

70.9 

86.8 

83.1 

85.3 

74.9 

2004 

80.9 

89.0 

82.3 

109.7 

73.5 

88.4 

82.1 

85.2 

75.2 

2005 

79.8 

89.3 

75.5 

92.4 

73.1 

87.2 

80.6 

84.4 

75.0 


The stark difference in the two retention rates can be seen in Figure 1 where the apparent full-time 
and full-time-equivalent retention rates are graphed for South Australia and Australia. Using the 
different retention rate indices, it is relatively easy to craft two very different arguments, with 
dramatically different implications for SACE. Based upon full-time retention rates, Crafter et al. 
(2006) argue that it is apparent that many students are not completing Year 12 studies, and SACE 
is failing somewhat dismally to hold students at school. Conversely, using full-time-equivalent 
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retention rates it could be argued that SACE is doing particularly well at enabling students to 
complete their education in a flexible fashion using part-time study. 
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Figure 1. Apparent Full-time and Full-time-Equivalent Retention Rates from Year 10 to 
Year 12 for South Australia and Australia (Source: ABS School Australia 2005, 
Cat. No. 4221.0) 

Crafter et al. (2006) are at least somewhat aware of the different stories available to them. 

Referring to the work of Lamb, Walstab, Teese, Vickers, and Rumberger (2004), they write: 

When the retention rate is adjusted to include part-time students, the Years 8-12 
apparent retention rate for South Australia rises by some 8 percentage points from the 
full-time rates shown above, to around 76 per cent, fifth highest in the nation and 
within 1-2 percentage points of the corresponding figure for Australia as a whole. This 
improvement in apparent retention reflects the large number of part-time students 
enrolled in Year 12 in South Australian government schools. (Crafter et al., 2006, 
p. 32) 


However, they refer to the large number of part-time students seemingly as an after-thought. In the 
main body of the Final Report, Figures 1.2, 2.1, 2.2, 2.4, 2.15, and Table 1.2 are all based upon 
full-time students. Only two figures, 2.3 and 2.5, in the main body of the Final Report use full- 
time-equivalent apparent retention rates. 

The Longitudinal Surveys of Australian Youth (LSAY) is a multi-wave study of cohorts that 
reached Year 12 typically in 1978, 1982, 1987, 1992, 1998, and 2001. Fullarton, Walker, Ainley, 
and Hillman (2003) using LSAY data reported Year 12 participation rates for South Australia in 
2001 to be 74 per cent. A potential limitation with the analyses undertaken by Crafter et al. 
(2006), Lamb et al (2004), and Fullarton et al (2003) is that they exclude students who leave 
school for at least a year and then decide to re-enter the education system. It is probable that the 
South Australian re-entry program is most likely responsible for the high full-time-equivalent 
retention rates. However, a fundamental question needs to be asked: If Crafter et al. (2006) are 
aware of the problem with full-time retention rates, why were those figures used to build their 
argument for a revised SACE? 


Chapter 1 of the Final Report is the leading chapter in the case for refonn. It refers only to full- 
time retention rates and does not mention full-time-equivalent rates (see Crafter et al., 2006, 
Figure 1.2) even though the authors acknowledge “South Australia has the highest number and 
percentage of part-time students in Year 12 in Australia” (p. 20). This chapter further compounds 
this line of reasoning by also presenting figures for people who are either engaged in full-time 
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study or full-time work even though South Australia has traditionally had the largest number of 
people who both study and work part-time (see Crafter et al, 2006, Table 1.2). The Review Panel 
presented the bulk of their argument using full-time retention rates while at the same time 
acknowledging that one of the desirable aspects of the current SACE is that it has the flexibility to 
enable part-time study practices. 

There is a substantial difference in retention rates across the school sectors in South Australia. 
Full-time and full-time-equivalent retention rates for non-government schools are consistently 
high (see Figure 2). For government schools, the full-time retention rates are typically between 60 
and 65 per cent while the full-time-equivalent retention rates approach those of non-government 
schools and are often above 85 per cent. That being the case, it is apparent that the government 
schools are offering an attractive part-time study alternative that is meeting the needs of a large 
number of people. This is most likely due to the adult re-entry high schools strategically placed 
across the greater metropolitan area (Ramsay, 2004). As noted by Lamb et al (2004): 

South Australia’s Re-Entry High Schools provide the clearest example of a second 
chance system. These are for students who have been out of school for at least six 
months. The minimum age of most students is about 16 years, but most students are 
aged 18 years or more. Some re-entry students are at the Grade 8 level academically 
and have to do bridging courses in order to be re-integrated into the mainstream 
curriculum. The nature of SACE (being unitised and open) lends itself to part-time 
enrolment. The average high school discourages part-time enrolment, so a lot of ‘re- 
entry’ enrolments are part-time students (p. 81). 


1998 1999 2000 2001 2002 2003 2004 2005 

Year 

Figure 2. Apparent Full and Full-Time-Equivalent Retention Rates for South Australia by 
School Type (Source: ABS School Australia 2005, Cat. No. 4221.0) 

The high full-time-equivalent retention rates in government schools certainly merit further 
exploration, starting with how retention rates are and should be calculated. Arguably, however, 
retention rates are a major problem for government schools and not for Catholic or Independent 
schools, and this problem is substantially addressed through re-entry schools. 

COHORT DEFINITIONS AND THE SACE REVIEW 

The retention rates reported by the ABS (2005, Cat. No. 4221.0) and Crafter et al. (2006) are 
additionally problematic due to the definition of the cohort being studied. A cohort is any group of 
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people with a common classification or characteristic (Johnson and Christensen, 2004). Cohorts 
are often used in panel studies where the same individuals are studied at successive points over 
time. Strictly speaking, the ‘student cohort’ presented in the SACE Review Report is not a cohort 
for a number of reasons, some of which are acknowledged by Crafter et al. (2006). Students 
moving from one schooling system to another, adults re-entering the schooling system, younger 
school leavers returning, an influx of international students in SACE, and grade retention all point 
to the changing nature of the cohort at the school, sector, and state level. The grade -based 
approach used by Crafter, Crook and Reid (2006) assumes (i) no grade retention and (ii) no net 
loss or gain of students from the school system. The first point is likely to be most problematic 
where schools implement a grade-promotion policy resulting in, for example, retention of Year 1 1 
students due to low academic perfonnance. The second point is related to the phenomenon of 
many students changing schools towards the end of their secondary education. Grade-based cohort 
calculations present an additional problem for SACE-type programs. Specifically, students who 
take two or more years to complete a full year of study impinge on the accuracy of the calculation 
and any meaningful interpretation of the statistic. 

Crafter et al. (2006) did not utilise any precise or suitable definition of cohort, thereby making 
their report limited, flawed, and misleading. Exploring the SACE-retention issue using Year 8 as 
the initial cohort and a non-equivalent Year 12 cohort four years later (see Crafter et al., 2006, 
Figure 1.2) is misleading since approximately 3 percent of the Year 8 students will leave school 
before the end of Year 10 (ABS School Australia, 2005, cat. No. 4221.0; see also Crafter et al., 
2006, Figure 2.1). The Year 10 cohort is a better indicator of student retention affected by SACE, 
but the progression of a student cohort would need to take into account the flexibility of the 
SACE. This flexibility means that SACE study can extend beyond the traditional Year 12 (see 
Crafter et al., 2006, Figure 2.5). Indeed, the flexibility of the present SACE makes it somewhat 
meaningless to talk about Year 11 and Year 12, especially when SACE study may commence in 
Year 10 and be completed beyond the traditional Year 12. Crafter et al. (2006) can be said to have 
‘manufactured a crisis’ by essentially overlooking one of the most prominent features of the 
present SACE (see also Berliner and Bruce J. Biddle, 1996). Without careful research into why 
South Australian students are leaving school shortly after they reach the legal school leaving age, 
it is impossible to say how the present or any future SACE is related to early school leaving. 

RESEARCH ON RETENTION 

Much research has been conducted on early school leavers. Referring to the work of SSABSA 
(1999), Cormack (2004), and Lamb et al. (2004), Crafter et al. (2006) argue: 

[T]he fact that the same pattern of withdrawal from the senior years of secondary 
schooling has been repeated each year for successive cohorts of students over an 
extended period of time suggests that systemic factors are operating to draw students 
away from full-time school education in the middle and senior years, (p. 37) 

What is noteworthy here is the acknowledgement, for the first and only time in the report, of the 
importance of the middle-school years. A substantial body of research, national and international, 
points to the middle-school years as being the time when most students make the decision to leave 
school (de Vries, 1993; Finn, 1989; Rumberger, 1995). Indeed, some researchers report that 
students start thinking of leaving school as early as Grade 1 (Alexander, Entwistle, and Horsey, 
1997). However, within the South Australian education system, they cannot legally do so until 
they are 16 years old. If South Australian students are making the decision to leave in the middle- 
school years, then the curriculum source of the retention problem is much better framed as a South 
Australian Curriculum Standards and Accountability Frameworks (SACSA) or middle-school 
issue. 
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Students decide to leave for a variety of reasons (de Vries, 1993; Dekkers and Claassen, 2001). 
Push factors, generated within the education system itself, include lack of ability, boredom or 
irritation, cultural/social isolation and/or discrimination, drug abuse, and detention. For example, 
fewer than 50 percent of middle-school low achievers are likely to complete school while almost 
90 percent of the very high achievers complete school (Marks and Ainley, 1997). Smyth, Hattam, 
Cannon, Edwards, Wilson, and Wurst (2000) showed that peer harassment is often a serious 
problem for at-risk students. Lamb et al. (2004) reported that some of the re-entering students 
were functioning at a Grade 8 level, indicating that there are systematic problems at the middle- 
school level and possibly earlier. 

Students who leave school for push factors “have an aversion towards school or attach very little 
importance to it for their future lives” (Dekkers and Claassen, 2001, p. 344). Research by Smyth 
and Hattam (2001; 2002) and Smyth, Mclnemey and Hattam (2003) shows that schools can 
contribute to, or ameliorate, the push factors through their culture. Focusing on the middle years, 
they argue that there are three types of school cultures, only one of which has substantial holding 
power. Gibson and Bejinez (2002) couch this in terms of high-quality student-teacher 
relationships. Ainley and Sheret (1992) report that “students who achieve at lower levels can find 
school equally satisfying in tenns of teacher-student relations, status, social integration, the 
relevance of the work which they undertake, and general satisfaction with school” (p. 72). 

School culture research shows that the holding power of schools extends well beyond the fonnal, 
academic curriculum. There is mounting evidence that participation in school sporting programs 
increases retention (Davalos, Chavez, and Guardiola, 1999). In South Australia, most non- 
government schools have compulsory competitive inter-school sporting programs. Similar 
sporting programs in government schools have diminished over the past two decades. Other 
research shows that “high school dropout is more prevalent in schools with low morale than in 
those with high morale” (Vallerand, Fortier, and Guay, 1997, p. 1 172). 

Pull factors are attractive factors external to schools, and include training courses and 
employment. The high full-time retention rates of the early 1990s can be attributed partly to the 
absence of pull factors. As noted by Lamb et al (2004): 

Declines in available jobs (measured by job vacancies) adds to school retention, as do 
generally poor labour market conditions, though this also varies by state and territory. 

The effects are strongest in South Australia indicating that falls in labour market 
opportunities had a strong impact on the rate of retention. The results, like those 
reported by Ryan (2003) and Karmel (1995), suggest that general labour market 
conditions have a marked effect on retention with deteriorating conditions adding to 
retention, (p. 126) 

McNeal (1999) reported that the type and intensity of part-time employment has a significant 
impact on student dropout. The suggestion is that students are in a position to choose the most 
attractive culture for their needs: work or school. 

Grafter et al. (2006) see the high full-time retention rates of the 1990s as a target: “However, it is 
clear that the government’s goal of restoring retention rates to the levels attained in the early 
1990s cannot be achieved by reform in the schooling sector alone. A broader set of policy 
responses is required.” (p. 33) That early 1990s high retention rate can be largely attributed to a 
global recession and a large decline in youth employment opportunities (Ryan and Watson, 2004), 
combined with the introduction of adult re-entry schools. While the government may seek to 
achieve the same high full-time retention rates through a revised SAGE, this seems unlikely 
without significant negative changes in the South Australian economy. Grafter et al. (2006) should 
have informed the government more fully of factors likely to mitigate restoring retention rates to 
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previous highs. Simply put, there is no statistical evidence presented in the Final Report that 
shows that the present SACE is responsible for students leaving school early. 

In a study of Dutch early leavers one year after they had left school, Dekkers and Claassen (2001) 
found that about half of the people had no regrets about their decision to leave early and 
approximately one-third admitted that it was not a sensible decision and they missed having a 
diploma. This latter group often left school because of push factors, while those who had no 
regrets left largely because of pull factors. De kk ers and Claassen (2001) characterise the Dutch 
school returnee as “often average students who were sometimes disruptive without having real 
behavioural problems” (p. 349). Enabling students who decide to leave school early to re-enter 
formal education at a later date may have significant motivational advantages. It would be 
worthwhile exploring the beliefs, attitudes, hopes, and aspirations of South Australian early 
school leavers using a longitudinal study design. Such a study would have been very useful in 
deciding which existing structures of the SACE support or mitigate students returning to school. 

By focusing on the SACE curriculum and assessment factors, and using inappropriate retention 
rates, Crafter et al. (2006) have crafted a very weak, and seemingly misleading, case for reform. 
They should have drawn more fully upon studies by Lamb et al who wrote that “Failure to 
establish meaning in the curriculum or to build satisfactory teaching relationships reduces the 
possibility of successful learning which is the most intrinsic motive for staying on at school” 
(Lamb et al, 2004, p. 10). Extending Lamb et al’s studies to take into account the influences of 
primary- and middle-school education on retention, Crafter et al (2006) should have explored the 
early- and middle-school experiences of South Australian students, studying how these 
experiences impact on the SACE experience. Further, they should have recommended the 
implementation of quality-of-instructional-experience measures and systems to monitor students’ 
achievement patterns as they progress through the school system (c.f. Lamb et al, 2004). 

RETENTION, TERTIARY ENTRANCE RANK AND ACADEMIC CREEP 

Crafter et al. (2006) argue that if the academic curriculum was substantively changed, it would 
enable (or entice) students to complete Year 12. In particular, they recommend that the Tertiary 
Entrance Rank (TER) be separated from the SACE (see Recommendation 21 in the report). The 
problem here is that the causal link between early school leaving and examination-orientated 
subjects or even the SACE is not well established. 

The argument from Crafter et al. (2006) basically assumes that there are inadequate curriculum 
provisions related to an ‘academic-dominated’ SACE curriculum. This is in spite of the large 
number of accredited SACE subjects from which students can choose. It is also in spite of the fact 
that Lamb and Vickers (2006) found that Year 12 non-completion rates “ranged from 28 per cent 
in schools where Vocational Education and Training (VET) counts towards the Year 12 certificate 
and there is a strong emphasis on workplace learning to 9 per cent in schools without any VET” 
(p. viii). Indeed, research indicates that schools with academically challenging curricula have 
substantially more holding power than schools with non-academically challenging curricula (Lee, 
Burkam, Chow-Hoy, Smerdon, and Geverdt, 1998). 

Crafter et al. (2006) state that “many respondents expressed concern that there had been a trend to 
make the content of subjects offered at Stage 2 in the SACE more abstract” (p. 62), and that the 
assessment requirements at Stage 2 had become more ‘academic’. What is unclear here is the 
processes used to make the subjects ‘more academic’. Marks, McMillan and Hillman (2001) 
report that South Australia has the lowest percentage of students of the Australian states receiving 
a Tertiary Entrance Rank (see Table 4). South Australia also has the second lowest percentage of 
its population who have a bachelor degree or higher (only Tasmania is lower) (see Table 5). Given 
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that New South Wales and Victoria have much higher full-time retention rates and also a higher 
proportion of their Year 12 graduation cohort earning university entry scores and successfully 
completing a foundational degree, surely an argument could be made to increase the academic 
thrust of the SACE? 

If academic subjects equate to tertiary-entry subjects, then in South Australia it is not clear that 
these subjects are dominating the curriculum. Perhaps instead the middle- and high-school 
curricula in some schools are not especially suited to enabling students to make transitions 
between stages of education? Regardless of the reasons for state-level differences in the 
percentage of student obtaining a TER, Crafter et al. (2006) appear to be exhibiting a degree of 
academic cringe. Rather than making the secondary and tertiary sectors work together, the 
proposed new SACE seeks to split them even further apart (see Recommendation 24 in the 
report). It is difficult to envisage how the separation of the TER from the SACE will facilitate 
university entrance for groups at risk. Actively encouraging and facilitating university pathways, 
especially among under-represented groups, may be a superior strategy. 

Table 4. Summary statistics for TER Scores, All Students and By Selected Jurisdiction 


(Source: Marks, McMillan, and Hillman 2001, p. 65) 


Standard Statistics (Weighted) 

All 


NSW 

Vic 

Qld 

SA 

Mean (Standard error) 


70.2 (0.5) 

69.1 (1.0) 70.9 (0.9) 64.9 (1.2) 

79.9(1.0) 

Median 


73.8 


71.1 

72.0 

70.0 

81.5 

Standard Deviation 


19.8 


22.5 

19.5 

24.1 

10.0 

Inter-Quartile Range 


31.3 


29.1 

31.2 

47.5 

22.0 

Percent of Sample with Score (Weighted) 

All 


NSW 

Vic 

Qld 

SA 

Of Year 9 Cohort 


52.6 


57.1 

62.6 

55.4 

45.0 

Of Year 12 Participants 


68.0 


73.0 

76.8 

70.5 

59.4 

Table 5, Level of highest educational attainment as percentage of 
(Source: ABS Education and Work, 2005, Cat. No. 6227.0). 

State total, 

State 

Year 10 
and below 

Year 11 to 
Year 12 

Cert, to 
Adv. Dip. 

Bachelors 
or above 

Total 

New South Wales 

29.5 


24.6 


23.3 

21.2 

100.0 

Victoria 

23.0 


32.0 


22.7 

21.0 

100.0 

Queensland 

28.7 


27.8 


25.8 

16.6 

100.0 

Western Australia 

26.8 


28.1 


25.4 

18.4 

100.0 

South Australia 

23.9 


34.9 


23.5 

15.7 

100.0 

Tasmania 

39.0 


21.3 


23.0 

15.3 

100.0 

Northern Territory 

22.4 


30.1 


28.0 

18.1 

100.0 

Australian Capital Territory 

17.5 


29.7 


18.4 

32.6 

100.0 

Australia 

27.0 


28.3 


23.8 

19.6 

100.0 


FULL-TIME RETENTION INDICES AND POSSIBLE RESEARCH 

The ABS apparent-full-time-retention-rate index is not suited to the purpose of the SACE review. 
Firstly, it is an ‘all or nothing’ index. That is, it does not take into account that completion of the 
SACE is an incremental task, with many students choosing to spread the task over three or more 
years. Second, students can decide to leave school for a number of reasons, including taking up 
apprenticeships. Since some of these reasons can and should be seen in a positive light, both from 
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student and societal perspectives, the interpretation of the full-time-retention-rate index will 
always remain problematic. Unless reasons for leaving are classified and incorporated into this 
index, such difficulties will remain and it will implicitly convey the understanding that leaving 
school for an apprenticeship or employment in the Australian Defence Forces is not seen as 
worthwhile. The full-time-retention-rate index used by the SACE Review Panel implies that 
formal, full-time school-based education is the only valid education. While this is in direct 
contradiction to the views put forward in other parts of the Report, the fact that Crafter et al. 
(2006) present major statistics and arguments based upon this index is highly problematic. A more 
effective approach may be a longitudinal school-retention analysis that uses individual student 
data, tracking them over time and school-type. Statistical techniques for such studies are well 
established but appear not to have been used in South Australia to date. 

The superficial analysis of the retention research by Crafter et al. (2006) does not do justice to the 
complexity of student-retention issues in South Australia. Since Catholic schools account for the 
majority of the non- government schools in South Australia, it would be worthwhile comparing the 
holding power of Catholic and government schools. Such analyses should take into account the 
nature of the family background, curricula, school culture and other variables. A fundamental 
question would be: Do Catholic and government schools with similar intakes have different 
retention patterns? If so why? The initial analysis would be largely statistical, while the follow-up 
question would almost certainly benefit from school-based research that incorporated micro-level 
analyses. 


SOCIO-ECONOMIC STATUS AND THE SACE REVIEW 

Socio-economic status (SES) is often interpreted as a proxy for the quality of the home learning 
situation (Raymond and Hanushek, 2003). SES is usually conceptualised as an attribute of an 
individual (or the person's family or household) that consists of different dimensions. Interest in 
SES stems in part from class-orientated research that “shows class as life condition is a powerful 
detenninant of all kinds of outcomes” (Sorensen, 1996, p. 1538). While some researchers fuse 
social and economic status, the terms are not synonymous (Williams, Williams, Kastberg, and 
Jocelyn, 2005). Wright (2002) reported that Weber centred his class analyses on life chances 
while Marx rooted his work in the problem of exploitation. Weber (1978) defined social status as 
“an effective claim to social esteem in terms of negative or positive privilege’” (p. 305). Weber 
(1978) also held that differential access to goods and opportunities for income is a defining 
component of economic class 

Socio-economic background, and its relationship to achievement, has been subjected to much 
research. A basic premise of these studies is that “schooling could be said to be fair if there is no 
impact of a student’s socio-economic background” on achievement and post-secondary education 
(Marks et al., 2001, p. 58). 

SES is often measured using objective criteria (Weiss and Fershtman, 1998). The SES index used 
by Crafter et al. (2006) is the ABS index of relative socio-economic disadvantage. This is the 
most general of the ABS indices, the others being the urban index of advantage, the rural index of 
advantage, the index of economic resources, and the index of education and occupation (Trewin, 
2001; 2002). The index of relative socio-economic disadvantage includes “all the available 
variables that either reflect or measure disadvantage” (Trewin, 2001, 2004, p. 1). The index 
components, shown in Table 6, include income and employment-status variables, and a race 
variable, the percentage of Indigenous Australians in the area. Thus, being Indigenous, and 
according to this index having too many Indigenous Australians in an area, is a disadvantage. 
While it may be that institutional racism is a very important social mechanism, there is nothing 
innately inferior or superior about being Indigenous, English, Irish, Italian or Chinese. And if there 
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are institutional mechanisms at play, then these should be documented in the formulation of the 
index. 

There are other concerns with the use of this index. For example, Williams et al. (2005) report 
that social status, and not economic status or some combination of the two components, is the 
stronger predictor of achievement. Another interpretive problem with the index is that “while the 
index may reflect an area’s disadvantage, it is not possible to say what aspects of disadvantage are 
being represented” (Trewin, 2004, p. 1). In census districts where there are pockets of advantage 
and disadvantage, the “pockets of advantage will not offset the pockets of disadvantage” and “low 
disadvantage does not equate to high advantage” (Trewin, 2004, p. 72). Trewin (2004) advises 
users looking for measures of advantage/disadvantage to use the other indices “depending on 
whether the user wants general advantage/disadvantage; economic advantage/disadvantage; or 
education and occupation advantage/disadvantage” (p. 72). The ABS index of relative socio- 
economic disadvantage incorporates the so-called ‘objective measures’ of social and economic 
status but can say little about the subjective evaluation of actions and traits at, for example, the 
school or class level. For example, economic class at the small group level shows “not a 
hierarchical totem pole of classes neatly stacked up on one another, but overlapping transactional 
circuits of vastly different scope and content” (Collins, 2000, p. 24). 

Table 6. List of variables in ABS Index of Relative Socio-economic Disadvantage (Source: 
Trewin 2004) 

1 % Persons aged 15 years and over with no qualifications 

2 % Families with offspring having parental income less than $15,600 

3 % Females (in Labour Force) unemployed 

4 % Males (in Labour Force) unemployed 

5 % Employed Males classified as 'Labourers and Related Workers' 

6 % Employed Females classified as 'Labourers and Related Workers' 

7 % One-parent families with dependent offspring only 

8 % Persons aged 15 years and over who left school at Year 10 or lower 

9 % Employed Males classified as 'Intermediate Production and Transport Workers' 

10 % Families with income less than $15,600 

11 % Households Renting (Government Authority) 

12 % Persons aged 15 years and over separated or divorced 

13 % Dwellings with no motor car at dwelling 

14 % Employed Females classified as 'Intermediate Production and Transport Workers' 

15 % Persons aged 15 years and over who did not go to school 

16 % Indigenous 

17 % Lacking fluency in English 

18 % Employed Females classified as 'Elementary Clerical, Sales and Service Workers' 

19 % Occupied private dwellings with two or more families 

20 % Employed Males classified as 'Tradespersons' 


Grafter et al. (2006) used the ABS index of relative socio-economic disadvantage, aggregated it to 
the local government area (LGA) level, and applied it to the location of school. They reported the 
correlation between full-time retention rates and socio-economic disadvantage in a graph, with the 
points on the graph indicating LGA (see Figure 3). They shaded in an area of the graph that 
excluded two LGA’s with above 100 per cent retention rates (Adelaide and Norwood). 
Acknowledging that “student movement across municipal boundaries distorts these geographical 
patterns” (p. 34), they claimed that there is a “strong, positive correlation between retention and 
socio-economic status [of where the school is located] in the metropolitan area” (p. 34). 
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The unweighted Pearson correlation coefficient between the LGAs and retention rates (excluding 
Adelaide and Norwood) is 0.80. However, when Adelaide and Norwood are included in the 
calculation, the unweighted coefficient drops to 0.65. Thus, the SES index apparently accounts for 
64 percent of the variation in full-time retention if Adelaide and Norwood are excluded, but only 
42 percent if they are included. The exclusion of Adelaide and Norwood without explanation and 
the use of apparent full-time retention rate is consistent with data-torturing methods. Data 
torturing involves manipulating the information in a variety of ways until the desired claim is 
established. 

LGAs are heterogeneous in terms of SES indicators. For example, the Port-Enfield LGA includes 
Port Adelaide (medium status), Osborne (low), Outer Harbor (high), Rosewater (low), and 
Semaphore (medium). So, while the Port-Enfield LGA has a medium overall SES, it is 
heterogeneous. The clustered, heterogeneous nature of the data distorts the correlation between 
SES and retention. 



Relative Socioeconomic 


Figure 3 Apparent retention rates, years 10 to 12, fulltime by socio-economic status of 
local government area and school location (Source: Crafter et al., 2006, Figure 
2 . 2 ) 

To illustrate the problem, consider a simulation study using 10 LGAs, each with 10 suburbs. For 
this study, the correlation between SES and retention rate was 0.84, marginally higher than that 
reported by Crafter et al. (2006). The graph for the simulated data is shown in Figure 4 and 
reflects the strong positive correlation. However, when the 10 local government areas are 
disaggregated, the correlation coefficient is 0.58. The scatter plot of the 100 simulated suburbs is 
shown in Figure 5. Bearing in mind that these two graphs (i.e., Figures 4 and 5) are based upon 
the same simulated data, it is readily apparent that different levels of aggregation provide 
somewhat different answers. The relationship between SES and retention rates would be expected 
to change again if we were to repeat the simulation using, for example, cohort or suburb block 
data. The problem with misleading correlations stemming from aggregated data has been known 
in education research since the Coleman report (Coleman et al., 1966) and is the major reason for 
the prevalence of multilevel-modelling techniques in statistical research today (Bryk and 
Raudenbush, 1992). Win and Miller (2005) recognised this analytical problem, saying that 
patterns between SES and achievement become distorted when the data are analysed at the 
aggregate level. Thus, the SES analyses presented by Crafter et al. (2006) are highly problematic. 

The local government areas are used to zone some schools and not others. Typically, students are 
expected to attend their local government school unless they can mount a case for attending one of 
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several magnet government schools. Magnet schools are government schools offering specific 
programs for students with, for example, high intellectual or musical potential. However, non- 
government schools are not generally limited to a specific district and it is common for this school 
type to draw students from across several LGAs. For example, the LGA of Adelaide with its high 
apparent retention rate has six non-govemment schools, several of which offer only Year 1 1 and 
12 education. These centralised, mainly non-govemment, schools draw students from many 
LGAs, with some students travelling well over 20 kilometres to attend a school. Correlating LGA 
SES with retention rates or achievement is therefore misleading, since the LGA of the school and 
the student do not always concur. 



Simulated 


Figure 4. Simulated apparent full-time retention rates and socio-economic status for ten 
local government areas (r = 0.84) 
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Figure 5. Simulated apparent full-time retention rates and socio-economic status for one 
hundred suburbs contained in ten local government areas (r = 0.58) 

International students are unlikely to be evenly spread across the education system. But a question, 
unanswered by the Review, is how these students are distributed. Their distribution will affect the 
apparent retention rates. While it is likely that international students will artificially raise the 
retention rate in some schools, this simply highlights the fact that retention rates and relationships 
between retention rates and SES, as presently calculated, are an insufficient basis for policy 
decisions. 
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In relating full-time retention rates to SES, Crafter et al. (2006) are arguing that dropping out is a 
SES problem (see also Hansen, Fisherkeller, and Johnson, 1995). This is unfortunate for at least 
three reasons. First, there has been considerable evidence to date that educational outcomes and 
SES are related to at least some degree in every country participating in international assessments 
in the past 40 years (Martin, Mullis, Gregory, Hoyle, and Ce, 2000). That is, regardless of the 
curriculum, education system, or society, the link remains, albeit in varying strengths. Even those 
countries that have policies designed to make schools ‘equal’ have a strong relationship between 
SES and achievement at the student level. There seems to be little support for the notion that the 
SES-achievement relationship will disappear. 

Second, correlations between variables are not enough since “policy cannot be intelligently 
conducted without an understanding of the mechanisms” (Deaton, 2002, p. 15). SES might cause 
higher educational achievement, education achievement may cause SES, or both might be 
correlated with other factors; indeed, all three possibilities might be operating simultaneously. If 
retention and educational attainment is affected by income, then education inequality may be best 
addressed by income-support schemes for the poor (Deaton 2002). Simply redistributing wealth, 
however, is likely to be problematic, not the least since gains by the poor will be offset by losses 
by the rich. If income is what matters for education, then its redistribution will only improve 
educational attainment if additional income has a lesser effect on education among the rich than 
among the poor (Deaton 2002). However, any policy would need to recognise that most of the 
variation in achievement and retention is likely to occur within a social or economic class, and not 
between classes. Thus, policy aimed at increasing retention or attainment should ideally be aimed 
at students at risk, not at particular social or economic groups. 

Third, even if the correlation between retention rate and SES can be interpreted as a causal 
relationship, it is by no means certain that changing the curriculum will address the supposed 
retention-rate problem. Power (1984), reporting on a South Australian study examining 
contributing factors to above-average school retention rates, questioned the extent to which 
curriculum refonn can lead to significant changes in retention rates. Indeed, given the high rate of 
part-time students in the government school sector, it appears that the present SACE curriculum is 
doing remarkably well in addressing the retention issue. It may be that a number of students are 
deciding to leave school for a short period, and then re-entering the system on a part-time, adult 
basis. Given the high apparent retention rates in Adelaide and Norwood, it would seem pertinent 
to conduct a study exploring the reasons why students seek an education outside their LGA, the 
degree of contentment with the decision, and the consequences of that decision. 

COMPLETION RATES AND THE SACE REVIEW 

Crafter et al. (2006) report that SACE completion rates appear “to have plateaued at a little over 
11 000 students a year” (p. 44). According to the accompanying graph in their Final Report 
(Figure 2.11), this represents, in 2003, 55.5 per cent of the Year 8 enrolments. Obviously, the 
method of calculation becomes important since they also report that 1994 and 1995 completion 
rates were 56.8 per cent and 56.6 per cent, respectively, whereas SSABSA reported “there was a 
substantial increase in the average SACE completion rate, from 63.3 per cent in 1993 to 76.4 per 
cent in 1994, and to 79.9 per cent in 1995” (SSABSA, 1999, p. 35). The respective completion 
rates that SSABSA reports for 2002, 2003, and 2004 are 86.1, 86.1, 87.5 per cent, respectively 
(SSABSA, 2005). The SSABSA rates are computed based upon the number of students entering 
SACE. That same SSABSA report notes that there are many reasons why students do not 
complete SACE. While the authors report low return rates for the questionnaire sent to students 
who did not complete SACE, there appears to be at least three clusters of reasons for not 
completing SACE. Reasons that were related more or less directly to the SACE curriculum, but 
may also be related to school curriculum structures included: (1) Subjects offered were not 
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relevant, (2) Not enough interesting subjects, and (3) Not enough subjects to choose from. Other 
reasons were more directly related to the school’s culture, such as (1) Teachers did not treat 
students fairly, (2) Did not like school, (3) Did not like teachers, and (4) Did not like school 
discipline. Other reasons more related to employment included: (1) Wanted to look for a job, (2) 
Started a TAFE course, and (3) Had a job. It is obvious that only some of the reasons for non- 
completion are likely to be addressed by curriculum and assessment changes in SACE. 

AVERAGE PUBLICLY-EXAMINED-SUBJECT SCORE AND 
STATISTICAL LOCAL AREA 

Crafter et al. (2006) report that students’ average scores in publicly examined subjects (PES) vary 
according to Statistical Local Area (SLA). SLAs are based on the boundaries of incorporated 
bodies of local governments where they exist. Crafter et al. (2006) state that higher scores are 
associated with parents being engaged in employment in managerial/administrative and 
professional occupations, female participation in workforce, and high family income. Similar 
findings have been reported in a large number of countries. This gives rise to the question of how 
the new SACE will change this. Further, what other variables are correlated with achievement? If 
teachers and schools make a difference in the education of students, then surely teacher and school 
variables will be related to achievement? That Crafter et al. (2006) do not present analyses using 
teacher and school variables may be interpreted as a tacit suggestion that teachers and school have 
no impact on student achievement. They claim “a range of academic and socioeconomic factors 
operate to produce a large degree of social differentiation in school leavers’ access to higher 
education” (Crafter et al., 2006, p. 46). Such a statement calls for a comprehensive explanation of 
how academic and socioeconomic factors determine achievement and access to higher education 
(see Cookand Campbell, 1979, p.32). 

While not forgetting that many students move across areas to attend school, the computation of an 
average score without an accompanying standard error of measurement is highly problematic. 
There is simply no way of detennining from the Final Report if any difference between regions is 
statistically significant or not. This is apart from the question of whether it is psychometrically 
appropriate to calculate means on essentially criterion-based ra nk s. A calculation of a mean 
assumes that a score of 8 out of 20 carries the same meaning across all subjects, and that the 
difference between 8 and 9 is substantively the same as 14 and 15. There is no evidence that this 
is the case. The calculation of mean scores across subjects are likely to be even more problematic 
given the large number of part-time students. For example, if a student spreads her studies over 
three years, what procedures are in place to ensure that an 18 in a subject carries the same 
meaning across those three years? 


CONCLUSION 

The Review Panel appears to have focused rather narrowly on its mandate “To achieve a 
curriculum and assessment framework that will meet the diverse needs of all students and result in 
high and more socially equitable levels of retention, completion and pathways beyond school” (p. 
9). It has, for example, failed to explore the reasons why students move schools at the end of Year 
10 that is evident in the growth of Years 1 1 and 12 in Adelaide’s private schools. It has failed to 
fully examine what keeps students at school, instead focusing narrowly upon two possible 
explanatory variables: SES and the so-called academic curriculum. 

The Pareto criterion applied to SACE is that the change from the present SACE to that 
recommended by the Review Panel will make at least one individual better off without making 
any other individual worse off. In their desire to make SACE more equitable they have largely 
ignored the successful student, and have instead exhibited a degree of academic cringe. Clearly, 
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Crafter et al. (2006) have failed to satisfy the Pareto criterion. Another way of evaluating the 
SACE Review is to utilise Social Choice Theory. Following the work of Sen (1993, 1995, 1999), 
social choice decisions should be based upon access to advantage, for example access to tertiary 
education. But it is not clear how the proposed new SACE, based upon insufficient and 
inadequate research, can deliver this. 

The SACE review panel did not build a sound case for its recommendations. South Australia has a 
high apparent full-time-equivalent retention rate, much higher than the apparent full-time 
retention rate. Given that SACE is designed to be flexible, something acknowledged by the 
Review Panel, it is incredulous that the panel then chose to build their case largely on full-time 
retention rates. Such an approach leaves them open to the charge of data torturing (Mills, 1993). 
Further, it is widely known that differences in achievement between schools can be attributed to a 
myriad of factors, extending well beyond socio-economic class and school location. For example, 
the Review Panel overlooked the difference in teacher experience that is apparent between South 
Australian regions. Another variable that may be important, but also overlooked, was whether 
service on SSABSA panels provides teachers with information not available to others. The 
statistical analyses presented in the report offer simple causal-effect relationships using aggregated 
data. Such relationships are largely inappropriate to education (Olson, 2004). As noted by 
Sadovnik and Semel (2001) “Over four decades of sociological research on the educational 
achievement of low-income children indicate that there is no one causal factor that explains 
academic underachievement, nor easy solutions to the problem” (p. 30). 

Educational research, like education, has a moral imperative. As education researchers, “we have 
a particular obligation and opportunity to take a leading role in seeing that the research that is 
done is truly good research” (Hostetler, 2005, p. 21). To allow, facilitate and enable the 
development and implementation of policies based upon naive, simplistic, incorrect, or inaccurate 
research is a clear violation of that imperative. The SACE Review Panel should have avoided one 
of the cardinal sins of statistical analysis: the tendency of “summary statistics to ... construct 
distorted and misleading pictures” is well known as the ‘ecological fallacy’ (Connolly, 2006, p. 
236; Troyna, 1984). The Review Panel’s use of aggregated retention, SES and achievement data is 
ultimately a fine example of how not to analyse educational data. That their statistical case for 
reform is substantively flawed suggests that the recommendations contained in the report should 
be treated with caution. 
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